Skip to content

Fix CUR 2.0 query with valid column names#114

Merged
Alexanderamiri merged 1 commit into
mainfrom
fix/cur-query-v3
Mar 27, 2026
Merged

Fix CUR 2.0 query with valid column names#114
Alexanderamiri merged 1 commit into
mainfrom
fix/cur-query-v3

Conversation

@Alexanderamiri
Copy link
Copy Markdown
Member

Summary

SELECT * is not supported by BCM Data Exports API — it returns ValidationException: Invalid QueryStatement.

Column names retrieved directly from the API:

aws bcm-data-exports get-table --table-name COST_AND_USAGE_REPORT \
  --table-properties '{"TIME_GRANULARITY":"DAILY","INCLUDE_RESOURCES":"TRUE",...}'

Key columns included: line_item_resource_id, line_item_product_code, line_item_unblended_cost, resource_tags (single map column in CUR 2.0, not separate resource_tags_user_* columns like legacy CUR).

Test plan

  • CI plan + apply succeeds
  • CUR export created in us-east-1 without validation error

SELECT * is not supported by BCM Data Exports. Column names retrieved
from: aws bcm-data-exports get-table --table-name COST_AND_USAGE_REPORT
with INCLUDE_RESOURCES=TRUE. Key difference from legacy CUR: resource_tags
is a single map column, not separate resource_tags_user_* columns.
@Alexanderamiri Alexanderamiri requested a review from a team as a code owner March 27, 2026 00:22
@Alexanderamiri Alexanderamiri enabled auto-merge (squash) March 27, 2026 00:23
@github-actions
Copy link
Copy Markdown

Terraform Plan

🚧 Changes detected — Plan: 1 to add, 1 to change, 1 to destroy.

Plan output
Acquiring state lock. This may take a few moments...

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
  - destroy

Terraform will perform the following actions:

  # module.cost_analytics.aws_bcmdataexports_export.cur will be created
  + resource "aws_bcmdataexports_export" "cur" {
      + id       = (known after apply)
      + tags_all = {
          + "environment" = "production"
          + "managed-by"  = "terraform"
          + "repo"        = "javaBin/platform"
          + "service"     = "platform"
          + "team"        = "platform"
        }

      + export {
          + export_arn = (known after apply)
          + name       = "javabin-cur"

          + data_query {
              + query_statement      = "SELECT identity_line_item_id, identity_time_interval, bill_bill_type, bill_payer_account_id, bill_billing_period_start_date, bill_billing_period_end_date, line_item_usage_account_id, line_item_line_item_type, line_item_usage_start_date, line_item_usage_end_date, line_item_product_code, line_item_usage_type, line_item_operation, line_item_availability_zone, line_item_resource_id, line_item_usage_amount, line_item_currency_code, line_item_unblended_rate, line_item_unblended_cost, line_item_blended_rate, line_item_blended_cost, line_item_line_item_description, product_product_family, product_region_code, product_instance_type, product_servicecode, pricing_unit, pricing_public_on_demand_cost, pricing_public_on_demand_rate, pricing_term, resource_tags FROM COST_AND_USAGE_REPORT"
              + table_configurations = {
                  + "COST_AND_USAGE_REPORT" = {
                      + "INCLUDE_MANUAL_DISCOUNT_COMPATIBILITY" = "FALSE"
                      + "INCLUDE_RESOURCES"                     = "TRUE"
                      + "INCLUDE_SPLIT_COST_ALLOCATION_DATA"    = "FALSE"
                      + "TIME_GRANULARITY"                      = "DAILY"
                    }
                }
            }

          + destination_configurations {
              + s3_destination {
                  + s3_bucket = "javabin-cur-553637109631"
                  + s3_prefix = "cur"
                  + s3_region = "eu-central-1"

                  + s3_output_configurations {
                      + compression = "PARQUET"
                      + format      = "PARQUET"
                      + output_type = "CUSTOM"
                      + overwrite   = "OVERWRITE_REPORT"
                    }
                }
            }

          + refresh_cadence {
              + frequency = "SYNCHRONOUS"
            }
        }
    }

  # module.dns.aws_cloudfront_distribution.sso_redirect will be updated in-place
  ~ resource "aws_cloudfront_distribution" "sso_redirect" {
        id                             = "E2F04LHEI6J2CL"
        tags                           = {
            "Name" = "aws.javabin.no-redirect"
        }
        # (20 unchanged attributes hidden)

      ~ default_cache_behavior {
          ~ target_origin_id       = "dummy" -> "s3-redirect"
            # (10 unchanged attributes hidden)

          - function_association {
              - event_type   = "viewer-request" -> null
              - function_arn = "arn:aws:cloudfront::553637109631:function/javabin-sso-redirect" -> null
            }

            # (2 unchanged blocks hidden)
        }

      - origin {
          - connection_attempts = 3 -> null
          - connection_timeout  = 10 -> null
          - domain_name         = "aws.javabin.no.s3-website.eu-central-1.amazonaws.com" -> null
          - origin_id           = "dummy" -> null

          - custom_origin_config {
              - http_port                = 80 -> null
              - https_port               = 443 -> null
              - origin_keepalive_timeout = 5 -> null
              - origin_protocol_policy   = "http-only" -> null
              - origin_read_timeout      = 30 -> null
              - origin_ssl_protocols     = [
                  - "TLSv1.2",
                ] -> null
            }
        }
      + origin {
          + connection_attempts = 3
          + connection_timeout  = 10
          + domain_name         = "aws.javabin.no.s3-website.eu-central-1.amazonaws.com"
          + origin_id           = "s3-redirect"

          + custom_origin_config {
              + http_port                = 80
              + https_port               = 443
              + origin_keepalive_timeout = 5
              + origin_protocol_policy   = "http-only"
              + origin_read_timeout      = 30
              + origin_ssl_protocols     = [
                  + "TLSv1.2",
                ]
            }
        }

        # (2 unchanged blocks hidden)
    }

  # module.dns.aws_cloudfront_function.sso_redirect will be destroyed
  # (because aws_cloudfront_function.sso_redirect is not in configuration)
  - resource "aws_cloudfront_function" "sso_redirect" {
      - arn                          = "arn:aws:cloudfront::553637109631:function/javabin-sso-redirect" -> null
      - code                         = <<-EOT
            function handler(event) {
              return {
                statusCode: 301,
                statusDescription: 'Moved Permanently',
                headers: {
                  location: { value: 'https://javabin.awsapps.com/start' },
                  'cache-control': { value: 'max-age=86400' }
                }
              };
            }
        EOT -> null
      - etag                         = "ETVPDKIKX0DER" -> null
      - id                           = "javabin-sso-redirect" -> null
      - key_value_store_associations = [] -> null
      - live_stage_etag              = "ETVPDKIKX0DER" -> null
      - name                         = "javabin-sso-redirect" -> null
      - publish                      = true -> null
      - runtime                      = "cloudfront-js-2.0" -> null
      - status                       = "DEPLOYED" -> null
    }

Plan: 1 to add, 1 to change, 1 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan"

LLM Review

Risk: 🟢 LOW

Plan adds CUR data export and refactors CloudFront SSO redirect from function-based to S3 website redirect, with no security or data loss concerns.

  • [routine] Creating aws_bcmdataexports_export for CUR data - new billable resource but standard cost analytics infrastructure
  • [routine] CloudFront function removal and origin ID change - refactoring redirect logic from CloudFront function to S3 website configuration, functionally equivalent
  • [routine] CloudFront distribution in-place update - only target_origin_id and origin configuration changing, no service disruption expected
  • 💰 [cost] New BCM Data Exports resource will incur charges for CUR data export, but this is intentional cost analytics infrastructure

@Alexanderamiri Alexanderamiri merged commit e97e369 into main Mar 27, 2026
5 of 6 checks passed
@Alexanderamiri Alexanderamiri deleted the fix/cur-query-v3 branch March 27, 2026 15:32
Alexanderamiri added a commit that referenced this pull request May 9, 2026
## Summary

`SELECT *` is not supported by BCM Data Exports API — it returns
`ValidationException: Invalid QueryStatement`.

Column names retrieved directly from the API:
```
aws bcm-data-exports get-table --table-name COST_AND_USAGE_REPORT \
  --table-properties '{"TIME_GRANULARITY":"DAILY","INCLUDE_RESOURCES":"TRUE",...}'
```

Key columns included: `line_item_resource_id`, `line_item_product_code`,
`line_item_unblended_cost`, `resource_tags` (single map column in CUR
2.0, not separate `resource_tags_user_*` columns like legacy CUR).

## Test plan

- [ ] CI plan + apply succeeds
- [ ] CUR export created in us-east-1 without validation error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant